Skip to content

test(e2e): Add a cloudflare-pi-durable end-to-end application - #25022

Merged
JPeer264 merged 3 commits into
jp/cloudflare-skip-pi-tablesfrom
jp/pi-durable-cloudflare-e2e
Oct 8, 2026
Merged

JPeer264 merged 3 commits into
jp/cloudflare-skip-pi-tablesfrom
jp/pi-durable-cloudflare-e2e

Conversation

@JPeer264

@JPeer264 JPeer264 commented Oct 3, 2026

Copy link
Copy Markdown
Member

Cloudflare now hosts pi-durable in Durable Objects through the Agents SDK PiHarness (changelog). This app runs pi-durable that way, with @sentry/cloudflare/vite as the only Sentry setup. It checks that a prompt becomes an invoke_agent trace with chat, execute_tool and provider http.client spans, that a throwing tool is reported on its span, and that a run resumes in a new trace after the Durable Object resets during a tool call. The reset uses ctx.abort(), which drops the object like an eviction; the next request starts it again, and PiHarness resumes pi from its SQLite state.

Like node-pi-durable and cloudflare-think, it calls a real model through OpenRouter, so it is optional. The changelog example uses Workers AI (createAI({ binding: env.AI })), but the AI binding needs a Cloudflare account even in local dev, so that path has a unit test in #24993 instead.

This app found a bug that #24993 now fixes: @sentry/cloudflare initializes the SDK inside each request, so the client is on the scope of that request only. The task phases ran on a copy of the default scope, had no client, and dropped every pi-durable span and error.

🤖 Generated with Claude Code

@JPeer264
JPeer264 added this pull request to stack #25004 October 3, 2026 16:26
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

⚠️ Warning: Base artifact is not the latest one, because the latest workflow run is not done yet. This may lead to incorrect results. Try to re-run all tests to get up to date results.

Path Size % Change Change
@sentry/browser 29.8 kB - -
@sentry/browser - with treeshaking flags 27.92 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 27.84 kB - -
@sentry/browser (incl. Tracing) 51.85 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 51.87 kB - -
@sentry/browser (incl. Tracing, Profiling) 54.8 kB - -
@sentry/browser (incl. Tracing, Replay) 91.58 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 80.45 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 96.29 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 109.27 kB - -
@sentry/browser (incl. Feedback) 47.32 kB - -
@sentry/browser (incl. sendFeedback) 34.84 kB - -
@sentry/browser (incl. FeedbackAsync) 39.95 kB - -
@sentry/browser (incl. Metrics) 30.82 kB - -
@sentry/browser (incl. Logs) 31.11 kB - -
@sentry/browser (incl. Metrics & Logs) 31.75 kB - -
@sentry/react 31.63 kB - -
@sentry/react (incl. Tracing) 54.17 kB - -
@sentry/vue 37.84 kB - -
@sentry/vue (incl. Tracing) 54.78 kB - -
@sentry/svelte 29.83 kB - -
@sentry/remix (Remix 3 client bundle) 56.83 kB - -
CDN Bundle 31.53 kB - -
CDN Bundle (incl. Tracing) 52.34 kB - -
CDN Bundle (incl. Logs, Metrics) 33.72 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 54.3 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 74.65 kB - -
CDN Bundle (incl. Tracing, Replay) 90.01 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 91.96 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 96.17 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 98.15 kB - -
CDN Bundle - uncompressed 92.92 kB - -
CDN Bundle (incl. Tracing) - uncompressed 155.46 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 99.46 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 161.41 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 229.5 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 275.63 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 281.57 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 289.33 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 295.26 kB - -
@sentry/nextjs (client) 56.53 kB - -
@sentry/sveltekit (client) 52.23 kB - -
@sentry/core/server 40.86 kB - -
@sentry/core/browser 13.71 kB - -
@sentry/node 149.8 kB +2.69% +3.92 kB 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 83.48 kB +0.18% +150 B 🔺
@sentry/node - without tracing 93.88 kB +0.21% +196 B 🔺
@sentry/node - without channel injection 127.97 kB +3.18% +3.94 kB 🔺
@sentry/aws-serverless 102.05 kB +0.15% +147 B 🔺
@sentry/cloudflare (withSentry) - minified 209.75 kB +0.02% +40 B 🔺
@sentry/cloudflare (withSentry) 520.07 kB +0.03% +129 B 🔺

View base workflow run

@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from 259167d to e5c842b Compare October 5, 2026 07:20
@JPeer264
JPeer264 removed this pull request from stack #25004 October 5, 2026 07:21
@JPeer264
JPeer264 changed the base branch from jp/pi-durable-e2e to jp/cloudflare-skip-pi-tables October 5, 2026 07:22
@JPeer264
JPeer264 added this pull request to stack #25029 October 5, 2026 07:22
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch 2 times, most recently from 8bfd4a7 to a095187 Compare October 5, 2026 11:22
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from a095187 to ae5a2f8 Compare October 5, 2026 12:39
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from ae5a2f8 to bf50db6 Compare October 5, 2026 12:46
@JPeer264 JPeer264 self-assigned this Oct 5, 2026
@JPeer264
JPeer264 marked this pull request as ready for review October 5, 2026 12:47
@JPeer264
JPeer264 requested a review from andreiborza October 5, 2026 12:47
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from bf50db6 to 0059f6c Compare October 5, 2026 12:53
@JPeer264
JPeer264 requested a review from a team as a code owner October 6, 2026 06:59
@JPeer264
JPeer264 requested review from isaacs and removed request for a team October 6, 2026 06:59
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from c57d751 to 0059f6c Compare October 6, 2026 06:59
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch 2 times, most recently from a528063 to 264b42d Compare October 6, 2026 07:55

@andreiborza andreiborza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we maybe also send the data to sentry and assert on it to make it truly e2e?

@@ -0,0 +1,133 @@
import { expect, test } from '@playwright/test';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: Let's use constants from sentry conventions as much as possible.

@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from 264b42d to 80d2a2d Compare October 7, 2026 09:05
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from 80d2a2d to 80540af Compare October 7, 2026 11:01
Comment on lines +75 to +76
this.ctx.abort('crash_once resets the Durable Object');
throw new Error('The Durable Object did not reset');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The crash_once function unconditionally throws an error after calling ctx.abort(). Since abort() doesn't halt execution, the tool will always fail instead of being reset and replayed.
Severity: MEDIUM

Suggested Fix

Remove the throw new Error('The Durable Object did not reset'); line. The ctx.abort() call is sufficient to trigger the reset, and the subsequent throw prevents the intended replay mechanism from working correctly. The code should simply call this.ctx.abort(...) and then exit the function.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location:
dev-packages/e2e-tests/test-applications/cloudflare-pi-durable/src/assistant.ts#L75-L76

Potential issue: In the `crash_once` tool, `this.ctx.abort()` is called to schedule a
reset of the Durable Object. However, `ctx.abort()` does not halt synchronous execution,
so the subsequent `throw new Error(...)` is always executed on the first invocation.
This causes the tool's `execute` function to fail with an exception, which is contrary
to the intended "replay-safe" behavior described in the code comments and expected by
the tests. The Node.js equivalent uses `process.exit()`, which correctly terminates the
process. This implementation will likely cause the tool to be marked as failed instead
of being replayed after the reset.

Also affects:

  • dev-packages/e2e-tests/test-applications/cloudflare-pi-durable/tests/pi-durable.test.ts:100~103

Did we get this right? 👍 / 👎 to inform future reviews.

@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from 80540af to a3e202a Compare October 7, 2026 13:51
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from a3e202a to d1084df Compare October 7, 2026 17:03
@JPeer264

JPeer264 commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

Could we maybe also send the data to sentry and assert on it to make it truly e2e?

Going to answer as here: #24994 (comment)

JPeer264 and others added 3 commits October 8, 2026 13:27
Runs pi-durable in a Durable Object through the PiHarness of the
Cloudflare Agents SDK, with @sentry/cloudflare/vite as the only Sentry
setup. The tests check the run trace with its chat, execute_tool and
provider spans, a throwing tool, and a run that resumes in a new trace
after the Durable Object resets during a tool call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JPeer264
JPeer264 force-pushed the jp/pi-durable-cloudflare-e2e branch from d1084df to c49ea1b Compare October 8, 2026 11:27
@JPeer264
JPeer264 merged commit 221dd86 into develop Oct 8, 2026
43 checks passed
@JPeer264
JPeer264 deleted the jp/pi-durable-cloudflare-e2e branch October 8, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants